Phase modulator redirecting incident light from darker areas to lighter areas of images

ABSTRACT

Undesirably high energy-consumption in illuminating devices can be caused at least in part by wasted (attenuated) light. Optically efficient illumination systems that may be used to project light, including images, use light redirection. Phase modulating devices may be applied to create desired light fields. Some embodiments provide dual or multiple modulation display or projection systems in which some or all of the modulators not only attenuate amplitude, but also amplify amplitude, or change the phase, frequency and polarization of the light provided by the light source.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Application No. 61/893,270filed 20 Oct. 2013. For purposes of the United States, this applicationclaims the benefit under 35 U.S.C. § 119 of U.S. Application No.61/893,270, filed 20 Oct. 2013 and entitled LIGHT FIELD PROJECTORS ANDMETHODS, which is hereby incorporated herein by reference for allpurposes.

TECHNICAL FIELD

This invention relates to light projectors. Some embodiments may beapplied to project images for viewing. Other embodiments may be appliedto create structured light for illumination or other purposes.Embodiments have example application in markets such as digital cinema,TV and home theater, portable and personal projection (military, mobile,etc.), indoor and outdoor personal and large screen advertising andinformation dissemination, signs/advertising/billboards/outdooradvertising, large-venue and live performance, medical imaging, virtualreality, computer gaming, office presentations and collaborative work,head-up displays in cars and other vehicles, smart illumination such asadaptive car head-lights, theatre spotlights, security/architecturallighting, high contrast planetarium projectors, indoor and outdoorgeneral illumination systems, street-lighting, road-lighting, aviationlighting systems, and high-contrast simulation displays such as flightsimulators

BACKGROUND

There are many situations where it is desired to create a light fieldthat has a specified luminance profile. Light projection systems have avery wide range of applications from architectural lighting to thedisplay of lifelike images. The projected light patterns can be dynamic(e.g. video), static (used for static images or static applications likethe beams of typical car headlights projected through a lens onto theroad, made by arbitrarily shaped optical surfaces, etc.). Light may beprojected onto a wide range of screens and other surfaces which may beflat or curved. Such surfaces may be fully reflective (like a canvasused in a cinema, a wall or a building) or partially reflective (such asthe windshield of a vehicle). Screens may be low-gain or high-gain,Lambertian or highly directional, high-contrast or lower in contrast.Light may be projected onto solid objects or onto a medium in a volume(such as fog).

Markets for and applications of light projectors include digital cinema,in-door and out-door advertising, medical imaging (both for display ofimages, as well as capture by a smart light source), large venue andlive events or performances, automotive heads up displays, carhead-lights and rear-lights, automotive entertainment and informationdisplays, home-theatre, portable business projection, television anddisplays for consumer applications, military applications, aviationapplications (like cockpit displays, smart landing-assistance,individual passenger entertainment displays), structured light sourcesfor industrial applications, automotive headlights and otherapplications.

Various devices may be used to spatially modulate light. These may becalled spatial light modulators (SLMs). Most SLMs provide a 2D array ofindependently and individually addressable pixels. Some examples of SLMsare reflective SLMs such as digital micro-mirror devices (DMDs), liquidcrystal on silicon (LCoS) devices and transmissive SLMs such as LCDpanels, transmissive LCD chips such as high-temperature polysilicon(HTPS) or low-temperature polysilicon (LTPS); and partiallyreflective/partially transmissive SLMs such as micro-electro-mechanicalsystems (MEMS) based systems in which some of incident light istransmitted and some of incident light is reflected. One problem is thatmost readily available spatial light modulation technologies aresubtractive. These SLM technologies operate by absorbing or removingundesired light. This contributes to the more general problem that lightprojection and often general illumination technologies tend to haveundesirably high energy consumption and may also have an undesirablylimited peak luminance.

Additional considerations apply to light projectors that are applied toproject images. For example, in such projectors raised black-levels,undesirably low contrast and limited colour-saturation can be concerns.

These limitations can mean that a dark viewing environment such as acinema, a dark living room, or some other lighting-controlledenvironment, is needed to get the best out of projected imagery. Thislimits the possible applications for projectors.

In all fields of technology including light projection achieving adesired level of performance at a competitive cost can be an issue.

There is a general need for light projection systems that ameliorate oneor more of the above-noted problems.

SUMMARY

This invention relates to systems that re-distribute light dynamicallyfrom a single or multiple light sources in order to achieve a desiredlight field (which may or may not comprise an image depending on theapplication) efficiently. Re-distribution of light involves taking lightfrom one area within a light field and directing the light to anotherarea within the light field. Some embodiments provide a controllableillumination system that can be used as a component in any of a widerange of lighting applications. Other aspects of the invention providemethods for creating light fields and light projectors which apply suchmethods.

A light field projection system according to some embodiments comprisesa data processor, a computer software program, one or more light sourcesand a light control mechanism that includes one or moredynamically-addressable optical elements. The light control mechanismmay also include one or more static optical elements such as lenses,mirrors, gaps, optical fibers, light guides and the like in an opticalpath. The software program, when executed by the processor may processdata specifying one or more desired target light fields (which may, forexample, comprise anything from desired headlight patterns to imageframes in a movie) and may cause the one or more dynamically-addressableoptical elements to redirect light to achieve the desired light fields.

Example embodiments of the invention provide light projectors,projection displays, methods for operating projection displays, mediacontaining computer readable constructions which, when executed by adata processor, cause the data processor to execute a method accordingto the invention, methods for displaying images, methods for processingimage data for display, methods for processing sequences of image datafor display, among others.

Further aspects of the invention and features of an illustrative set ofexample embodiments are illustrated in the accompanying drawings and/ordescribed in the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate non-limiting example embodiments ofthe invention.

FIG. 1 is a block diagram of an example system (light efficientilluminator) at a high level.

FIG. 2 is a block diagram of the system of FIG. 1 at a more detailedlevel showing basic components of a light efficient illuminator.

FIG. 3 is a block diagram detailing the components within the lightmodule which is a component of the light efficient illuminator shown inFIG. 2.

FIG. 4 details example optical components within the light module ofFIG. 3 and also depicts example light profiles at each stage within thelight module.

FIG. 5 is a block diagram of an example dynamically-addressable lightredistribution module.

FIG. 6 shows two examples (top and bottom) of light profiles at thelight module (left), at a dynamically-addressable focusing element(center) and at the system output.

FIG. 7 illustrates example optics of a simple system that does notrequire (but is compatible with) coherent light.

FIG. 8 illustrates example components of a system that re-distributeslight and utilizes a clean-up stage (e.g. an array of integration rods)as well as showing example light profiles at each component.

FIG. 9 is a block diagram illustrating operation of anoptimization-based method for determining phase shifts to apply todifferent areas of a dynamically-addressable focusing element to createa target light field (or iterative program required to operate thesystem in ‘exact mode’).

FIG. 10 is a block diagram illustrating a procedural method for creatingan approximation to a target light field.

FIG. 11 shows three examples of an optical setup for use with coherentlight.

FIG. 12 shows example optics for a variable laser scanning approach.

FIG. 13 shows example optics of another variable laser scanning approach(with dynamically focusable lenses or other optics).

FIG. 14 is an example of an optical switch using MEMS (microelectromechanical systems).

FIG. 15 is an example of how an optical switch or other lightre-distributor can be used in concert with a clean-up module (e.g. anarray of integrating rods).

FIG. 16 is a block diagram of a system including a refinement module(e.g. a light efficient illuminator in combination with a spatial lightmodulator (SLM).

FIG. 17 shows example optics of a system including a refinement module.It also depicts example light profiles at the output of the lightefficient illuminator and at the input of the imaging device (e.g. SLM).

FIG. 18 is a block diagram of an illumination system including aclean-up stage and a refinement stage.

FIG. 19 shows example optics of a system including a clean-up stage anda refinement stage. It also depicts example light profiles at eachstage.

FIG. 20 shows intensity-time and intensity-location plots of atime-multiplexed or color field sequential system. Two example methodsuse two different timing schemes.

DESCRIPTION

Throughout the following description specific details are set forth inorder to provide a more thorough understanding to persons skilled in theart. However, well known elements may not have been shown or describedin detail to avoid unnecessarily obscuring the disclosure. The followingdescription of examples of the technology is not intended to beexhaustive or to limit the system to the precise forms of any exampleembodiment. Accordingly, the description and drawings are to be regardedin an illustrative, rather than a restrictive, sense.

FIG. 1 illustrates schematically an example system for generating alight field according to an example embodiment. The system is configuredto receive data describing a target light field and to produce an outputlight field that closely matches the desired target light field byredirecting light. In some cases the output light field may requirefurther optical or other treatment to produce the desired target outputlight field. Such treatment may be provided by one or more refinementstages as described herein. As described in more detail below, thesystem does this in an optically efficient manner, in which little lightis wasted. The system includes one or more light generators (lightsources) that can be operated to create light. In some embodiments thesystem operates the light generators to output light and then redirectsthe light to provide the output light field. The output of the lightgenerators may be controlled to match the amount of output light to theamount of light required for the output light field. Most of the lightgenerated by the light generator(s) may end up in the output lightfield.

FIG. 2 depicts an example dynamic light-efficient illuminator in moredetail. A program is executed on a data processor. The program receivesdata describing the target light field and computes a lightredistribution scheme to be applied by a dynamically-addressable lightre-distributor. The program also computes data describing theintensities for one or more light sources that generate light suppliedto the light redistributor.

A data processor is not required in all embodiments. In some embodimentslight redistribution schemes for a predetermined set of light fields aredetermined in advance. In such embodiments data defining the lightredistribution schemes corresponding to the light fields may be stored,embodied in fixed or configurable logic circuits or the like.Appropriate data can then be applied to control the light redistributorto generate a desired one of the light fields without the necessity forany computation.

The light module (comprising the light sources and light redistributorand associated optics) receives power, as well as data describing theintensities of one or several light sources and generates a light fieldhaving a desired light profile. An example light module is furtherillustrated in FIG. 3.

FIG. 3 is a more detailed schematic view of an example light module. Thelight module comprises a data processor, a power regulator, one or manylight sources, and a light combining and beam-shaping module.

The data processor receives data describing the intensity for the lightsource(s) and produces a control signal for the power regulator that inturn controls the amount and/or timing of power distributed to eachlight source. The power regulator may include a plurality of separateoutputs and/or may include a plurality of independent power regulationcircuits.

The light sources may be of any of a wide variety of types. One lightmodule may optionally include light sources of a plurality of types.Some examples of light sources are: lasers, arc-lamps, LEDs,high-intensity lamps, etc.

Each light source may emit light of different shapes, intensities andprofiles. An example light profile produced by a light source could be auniform, rectangular intensity profile which could be produced usingintegrating rods or other optics. Another example of a light profile hasa Gaussian or sum of Gaussians intensity profile. Another example is anarray of rectangular uniform profiles (blocks) with differentintensities. In another example, the light profile produced by the lightmodule can take any desired shape.

The light from the light source(s) is then coupled to the input of thelight redistributor. The coupling may involve spatially combining lightfrom a plurality of light sources and/or shaping the light optically toyield light having the desired light profile for input to thedynamically-addressable light redistributor. The light combination andbeam shaping may be done using common optical elements such asbeam-splitters, polarizers, wave plates, multi-layer thin films,beam-combiners, micro-lens arrays, lenses, apertures and/or mirrors.These elements can be optimized for the nature of the light emitted bythe light sources (e.g. narrow-band or broad-band light sources).

In one preferred embodiment, light from a plurality of light sources iscoupled into a common optical fibre and at the output of the fiber thelight is collimated using a set of lenses.

Data present within the system, for example the control signal or datadescribing the light profile incident on the Dynamically-addressableLight Re-Distributor may be provided to external components or systems,for example in the form of metadata.

FIG. 4 shows example light profiles at various stages through the systemillustrated in FIG. 3.

The light redistributor may controllably alter the nature and/ordistribution of light using techniques that are not primarilysubtractive. For example, the light redistributor may exploitinterference of electro-magnetic waves (light), to modulate thedistribution of light by controlling its phase characteristics and/ormodulate the frequency of the light in order to change the apparentcolour of light. Both of these examples show how light can be changedwithout converting energy from the light into wasted heat by absorbingthe light.

In one embodiment one or more light sources LS1 to LSn are coupled intoone or more optical fibres 408, for example using focusing lenses 409.405 shows an example light profile of the n^(th) light source. Thecombined output from the optical fibres 408 are relayed onto thedynamically-addressable light redistributor 407, for example using relaylens system 400, comprising, for example, two focusing lenses 401 and402. The combined effect of the two lenses 401 and 402 in 400 may be toasymmetrically magnify the output profile from the optical fibres 408.403 shows two example plots: at top the combined intensity across onespatial dimension of 407 and at bottom contours of the light profilepresent on 407 in two dimensions. 404 shows an example of the same typeof plots of the intensity and contours present at the output of 408.Furthermore the top plot in 404 illustrates how the total intensity ofthe light profile may be made up from a plurality of light profiles forexample 406 from each light source LS1 to LSn.

FIG. 5 schematically illustrates one type of dynamically-addressablelight redistributor. The dynamically-addressable light redistributorcomprises receiving optics, and a dynamically-addressable focusingelement.

Examples of devices suitable for use as dynamically-addressable focusingelements in different embodiments include: transmissive 2D arrays ofcontrollable liquid crystal compartments with the property that thecompartments can be controlled to selectively retard the phase of light,effectively causing a change in path-length. The same type of devicecould be implemented in a reflective manner. A dynamically-addressablefocusing element may also affect the polarization of light. Some devicesmay alter several light properties simultaneously.

In some other embodiments a dynamically-addressable focusing elementcomprises one or more scanning mirrors, such as a 2D or 3Dmicroelectromechanical system (MEMS); and/or. one or more deformablelenses or mirrors or other optical elements. A dynamically-addressablefocusing element may also or in the alternative comprise one or moreoptical switches.

The receiving optics transforms an incoming light profile from the lightmodule into an illumination light field that matches or approximatelymatches the size, shape and angular acceptance range of thedynamically-addressable focusing element. The receiving optics could,for example, comprise one or more of: a prism system, a lens, a freespace optical path, an integrating rod or waveguide.

The dynamically-addressable focusing element is controlled by data thatcorresponds to a light re-distribution scheme. The data may describe avariation in light path-length across the device which, when implementedby or executed on the dynamically-addressable focusing element causesthe formation of the desired output light-field.

FIG. 6 shows some examples of light profiles created in the light moduleand an example final system output light profile. FIG. 6 includes twoexamples of what the light fields may be at three stages of FIG. 5: anincoming light profile from the light module, the illumination lightfield present at the dynamically-addressable focusing element, and thelight field going to the output optics. Example 1 illustrates the casewhere a fairly uniform input light field is transformed into anarbitrary and desired output light field. Example 2 illustrates the casewhere a non-uniform input light profile made up with light from an arrayof independent light sources is transformed into a uniform light fieldat the output.

FIG. 6 shows two example sets of light profiles at various stages of theLight Efficient Illuminator. In 610, an application is shown in whichthe light profile 611 at the output of the light module is transformedinto an arbitrary light profile 613 at the output of the light efficientilluminator. The arbitrary light profile 613 may directly represent thetarget light field or target image. The light profile 612 shows thelight profile 611 relayed and magnified onto the dynamically-addressablefocusing element. M1 and M2 represent the magnification of the opticalsystem, resulting in the intensity scale factors N1 and N2,respectively.

In 620, an application is shown in which the light profile 621 at theoutput of the light module is transformed into a uniform light profile623 at the output of the light efficient illuminator. The light profile622 shows the light profile 621 relayed and magnified onto thedynamically-addressable focusing element. M1 and M2 represent themagnification of the optical system, resulting in the intensity scalefactors N1 and N2, respectively.

Providing a module capable of yielding a wide range of output lightfields is advantageous as such modules may be optimized for opticalefficiency and may be applied in any of a wide range of applications asdescribed above. Apparatus according to the invention may also beintegrated directly into projectors, displays, lights etc.

Explanation of how to Use a Phase Modulator to Create a Desired LightField, Description of Basic Arrangement.

Example phase modulation devices include:

-   -   Spatial light modulators (SLM), for example a 1D or 2D array of        pixels, in which the drive level addressed at a pixel correlates        to the phase delay applied to the light impinging on that pixel,        for example the drive levels between 0 and 65535 may correspond        to the range of phase delay between 0 and 2π radians (one cycle        of the wavelength of the light).    -   Such a spatial modulator can simultaneously change the state of        polarization of the light (an example is a transmissive        liquid-crystal display, or a reflective liquid        crystal-on-Silicon display (LCoS)). Alternatively, such an SLM        can be designed ONLY to affect the phase delay of that pixel,        but not its polarization.    -   An acousto-optical modulator (AOM; also called a Bragg cell) can        affect deflection angle of the incoming light, its phase,        frequency and polarization characteristics.    -   A grating light valve (GLV); currently, these devices are 1D        addressable arrays where each pixel or element can vary the        phase of the impinging light by mechanically varying the path        length.        How to Use a Phase Modulator to Create a Desired Light Field:

A lens in the classical sense is a variable thickness piece of glassthat retards the phase of the incident light differently across the lenssurface, resulting in a focused or de-focused spot of light depending onthe curvature or shape of the lens. A similar effect can be achieved byretarding the phase of the incoming light beam using a phase modulatingdevice (PMD). For example, the effect of a lens can be achieved byaddressing a varying phase pattern on the PMD, with for example 2π phaseretardation in the centre of the PMD, falling off to 0 phase retardationon the edges of the PMD. Stronger lenses (lenses with a shorter focaldistance) can be achieved by controlling the PMD to provide phasemodulation in a pattern like that of a Fresnel-lens.

A PMD may be controlled to simulate the effects of other opticalelements, such as prisms and gratings in a similar fashion, for exampleby applying a slowly varying amount of phase delay in one direction onthe PMD.

Different effects may be combined on the PMD. An example is aphase-pattern that both focuses and shifts the incoming light profile.This may be achieved by controlling the PMD to alter the phases of lightin a pattern that superimposes (adds) the respective phase delaypatterns for a lens and a prism.

Several lenses side-by-side or super-imposed on the PMD can coarselyapproximate an image. By suitably controlling a PMD to emulate theaction of a number of lenses one can create a situation where an imageor parts of an image are in focus anywhere along the direction of lightpropagation, for example in several planes.

More complex images or illumination profiles can be achieved bycontrolling the PMD to present a phase adjustment that iscontinuously-varying over the area of the PMD as opposed to controllingthe PMD to emulate a combination of discrete optical surfaces such aslenses and/or prisms.

Example Arrangements for Light Sources

Types of Light Sources:

The light source for the system can for example be one or more lasers,arc-lamps, LEDs, or even the sun. The specific characteristics of alight-source can make it more desirable than others. For example, alaser might be preferable over a broad-band lamp due to its small beamsize, limited optical spread (resulting in a very high intensity),limited etendue, its narrow spectral-band frequency distribution (andthereby pure colour), its polarization-, lifetime-, decay-, efficiency-,coherence and collimation characteristics.

Example Optics to Bring Light to/from Phase Modulation Device:

Light needs to be transported to and from the phase modulating device(PMD). It may also be desirable to have one or several of itscharacteristics changed, like its illumination profile, itsmagnification and shape, its polarization or frequency. It is sometimespreferable to have the incoming light profile match the shape of thePMD. Examples of elements or devices that can be used to achieve thisinclude but are not limited to:

-   -   Beam expanders consisting of one or several physical lenses;    -   Single or bundled optical fibres;    -   Prisms;    -   Cylindrical lenses;    -   Micro-lens arrays;    -   Gratings;    -   Diffusers;    -   Polarization filters;    -   Apertures;    -   Wave-plates;    -   Integrating rods.        Different Light Sources Illuminating Different Areas of        Modulator

In the case that several light sources are used to illuminate the PMD,the light profiles from these separate sources may overlap, partiallyoverlap, or not overlap on the PMD.

It might be desirable to achieve a uniform combined light profile on thePMD. In that case, optical elements such as integration rods, micro-lensarrays, diffusers, or other light-shaping devices may be included touniformly illuminate the PMD.

The intensity of one or more of these light sources may be independentlyadjusted in order to achieve a desired combined light profile on thePMD. As an example, one might want to have the incident light profile bebrighter in the centre of the PMD than on its edges.

Alternatively, the total amount of light from one or more light sourcesincident on a region on the PMD over a fixed period of time can beadjusted by turning the light-sources on and off (pulse-widthmodulation, PWM) instead of adjusting the intensity of the lightsources. The on-time relative to the off-time determines the overallamount of light incident on the PMD over a period of time. The durationof the on-pulses in the time period may vary, and they may be periodic,non-periodic or random; the total on-time may be more relevant toperceived brightness or colour by an observer.

The pulsed PWM approach may require a synchronization between theincident light sources and other elements in this optical system.

Example Projection Arrangements

The most simple projection system comprises a light source, somebeam-shaping optics to partially or fully illuminate the PMD, and aphase-retardation pattern on the PMD.

In another embodiment, a plurality of light sources may illuminate thePMD.

In another embodiment, the output light field from the PMD may berelayed onto a spatial light modulator (SLM) for further refinement, forexample by amplitude modulation.

In another embodiment, the output light field from the PMD may bepresented onto one or more integration rods, each of which integratesall the light incident on it into a uniform output. This output in turnmay be relayed onto different regions of the SLM for further refinement.

In some embodiments, it may be desirable to relay the output light-fieldfrom any of these system arrangements onto a projection screen orsurface.

Different Focusing Arrangements

In one common application, it is desirable that the projection systemform an image on a flat, a near-flat, or at most slightly curvedsurface. In other applications, it may be desirable to project onto oneor several non-flat surfaces or screens, for example a curved screen ora rough surface like a brick-wall, or a building. In anotherapplication, one might wish to project onto a moving object, like aperson or animal, or a vehicle moving either on land, in the air, onwater or submerged under water. Such an application may require thesynchronized capture and detection of that moving object. It may also bedesirable to project images or patterns onto different objects within avolume, or to different points in a volume. For example, in a carhead-light application, one may wish to illuminate an upcoming stop-signvery brightly, but an approaching vehicle with reduced intensity.

A phase-pattern present on a PMD can be arranged in such a way as tofocus at different depths or with different effective focal lengths. Thecorresponding light field can include images or features that are infocus at one plane or point, and other images or features that aresimultaneously or time-delayed in focus at another plane or point. Insome embodiments, one can arrange the phase-pattern on the PMD so thatthe system is focus-free, or in focus at any distance.

Focus at any of these points or planes can be achieved using a phasepattern on the PMD exclusively (“dynamic lenses”), rather than withphysical lenses. It can also be achieved using a combination of dynamiclenses on the PMD and physical lenses. It can also be achieved usingonly physical lenses.

How to Image in Colour.

In one embodiment, colour images are formed by mixing appropriate ratiosof light of three differently perceived colours, for example red, greenand blue everywhere in the target image. For example, in an imagedescribed by a pixellated 2D array, each pixel may have differentamounts of red, green and blue contributions. In another embodiment,light of four or more differently perceived colours may be mixed for asimilar effect.

In one application, the different colours can be supplied simultaneouslybut in parallel, for example by shining each of the primary colours ontoits own SLM or PMD, and then recombining the different colours using acolour re-combiner, for example a cube made up from dichroic mirrors (aso-called “X-cube”). In another application, the three (or more) lightsources may illuminate different areas of one PMD. Because of thesteering-capability of the PMD(s), these three (or more) regions may becombined downstream in the optical system by applying the appropriatephase pattern on the PMD(s). The phase-patterns for the differentlycoloured regions will be determined individually in order to ensure thatwhen they are combined, they form a colour-image as close as possible tothe target-image.

In another application, each frame of the video is broken down into itsseparate colour channels, for example red, green and blue. Thedifferently coloured light sources may illuminate one image-formingdevice (such as the PMD, or a conventional amplitude-modulating SLM) insequence (so-called time-multiplexing). It is desirable for thesesequential colour fields to be presented in fast succession, so that thehuman visual system perceives the resulting colour image without colourbreak-up or flicker. An example is to show a red colour field for thefirst image of a video sequence for 1/72th of a second, followed by thegreen colour field for the first image for the next 1/72th of a second,followed by the blue colour field for the first image for the third1/72th of a second for an effective frame-rate of 24 Hz or 24 frames persecond. This process is then repeated for the ensuing frames of thevideo or image sequence.

In one embodiment, time-multiplexing may be combined with beam-steeringin the following manner. In the case of a three-primary imaging system(for example R, G, and B), each of the three colours may betime-multiplexed by different amounts, providing a different amount ofintegrated intensity of R, G, and B onto the display screen. Forexample, within one complete frame-duration, the on-time of the redcolour field may be a much larger fraction of the total frame-durationthan the green colour field on-time. This can be extended to a highernumber of primary colour light sources.

In another embodiment, the different colours may be providedsimultaneously off one imaging device per colour channel. Dimming ofeach colour channel independently may be achieved by multiplexing eachcolour channel.

In another application, broadband light or several combined narrowbandlight sources may be split apart using a diffraction grating or a prism(for static deflection), or a PMD (for dynamic deflection). Thechromatically split light may be relayed onto a (second) PMD whereseveral bands of colour may be processed independently in order to forma final colour image at the display screen.

In one or all of the above-mentioned applications, one might useintegration rods to make the intensity profile of different regions moreuniform before further processing.

Control Modalities

Examples of Control System Hardware

The control system hardware may comprise a data processor and computerreadable data representing the sequence of light fields (“targetimages”). In some embodiments, it may be preferable to process the datain real-time, and in other embodiments the data may be pre-processed andstored on computer-readable media. In some applications, the dataprocessing may be executed on a field-programmable gate-array (FPGA), anapplication-specific integrated circuit, or a general purpose possiblyCPU-optimized or possibly GPU-optimized computer.

A PMD may be controlled to create any of an exceedingly broad range ofoutput light fields. Determining what data to use to drive the PMD toyield a specific desired output light field may be done in a number ofways. In a more-computationally expensive but more exact mode one canapply a mathematical model of the inverse transformation provided by theentire optical system (including the PMD) to start with the desiredoutput light field and calculate the PMD pixel settings corresponding tothe desired output light field. A less-computationally intensive butless exact mode includes setting parameters for one or more opticalelements (lenses, prisms, etc) that can be emulated by the PMD to yieldan output light pattern that approximates the target light pattern. Theparameters may, for example, include sizes, locations and opticalstrength.

In one embodiment, the applied phase pattern on the PMD is determined sothat given the particular optical layout of this imaging system and amatching model of this system, the light applied to it will result in animage on the display screen that very closely matches the desiredlight-field (the “target-image”) provided by the content-device (forexample from a TV network, a DVD, a Blu-Ray disk, or from a streaminginternet source). Such a phase-pattern can be achieved by an iterativeoptimization approach, where the difference between the current solutionand the target-image is iteratively minimized until a completioncriterion is satisfied.

In another embodiment, the applied phase pattern on the PMD isdetermined so that the resulting light-field output from the PMD onlypartially matches the target-image or target light-field. As an example,an approximation of the target-image may be formed by summing up anumber of similar or dissimilar features each described by a set offeature parameters. The features can for example be a set of Gaussianintensity profiles placed at different positions, with differentamplitudes and different full-width half-maxima. This may result in alow-pass filtered or blurred version of the target-image, but may alsobe executed at reduced computational cost. In this embodiment, a“refinement-stage” may be required, and the output light-field of thissystem may be relayed onto for example a secondary amplitude modulatingspatial light modulator or another PMD. The purpose of the refinementstage in this situation would be to restore the finer details of thetarget image or light-field.

In another embodiment, one could simulate basic physical-optics devicessuch as lenses and prisms on the PMD in order to scale and translateregions or parts of an image. Scaling of a particular input lightintensity profile may be obtained by simulating a lens on the PMD. Thislens may have a circular or a two-dimensional cylindrical symmetry.Translating a particular input light intensity profile may be achievedby simulating a prism on the PMD, for example by gradually varying thephase delay across the PMD. A lens may be described simply by itscurvature, and a prism may be described by its tilt relative to theoptical axis. The result of applying a uniform light distribution onto alens is to focus, or de-focus it. If the display screen is notpositioned exactly at the focal distance of said lens, a round shape ofuniform light is created if the lens has circular symmetry, and arectangle is created if the lens has a cylindrical shape.

In some embodiments, one might wish to simultaneously scale andtranslate a region of the image. This can be achieved by superimposingthe simulated patterns for a lens and a prism.

Should the desired lens or prism require an amount of phase retardationthat the PMD is unable to provide, then one could utilize a Fresnelapproach where an integer multiple of 2π (is subtracted from the desiredvalue. For example, a phase retardation of “π” can have the same effecton the light-steering capabilities of a PMD as can a phase retardationvalue of 5π. The equivalent mathematical function may be the modulus; inthis example modulus (5π, 2π)=π.

Ways to Determine Areas;

In one embodiment, the light redistribution scheme describes a method todetermine how one or more regions of light entering a lightredistributor could be mapped to other regions in the light exiting thelight redistributor. As an example, the scheme may describe how thelight incident on a 10×10 pixel region could be redirected to a 30×5pixel region, possibly translated to another part of the image. In thisparticular example, the incident light used to be illuminating only 100pixels (10×10), but after the mapping illuminates 150 pixels (30×5).Since the illuminated area in this example has been increased by 50%,the resulting intensity has been reduced by one-third. The lightredistribution scheme may analyze the target-image in order to arrive athow a particular region is to be mapped.

In another embodiment, a number of equally sized regions gets mapped tothe same number of regions, but these regions may all be of differentsizes. Such an approach can be utilized to create a low-resolutionversion of the target image or light-field. The high-resolution parts ofthe image will now have to be restored in a downstream modulation or“refinement” stage, for example by an amplitude modulating SLM.

In some embodiments, the amount of scaling and shifting that getsapplied to the equally sized regions of the image correlates in a director indirect way to the total intensity of light required by the targetimage or light-field in that region.

In other embodiments, the target image is analyzed and a number ofdifferently sized regions are mapped to the same number of equally sizedregions. The output of this approach may be incident on an array ofintegration rods matching the size and shape of each region.

In another embodiment, differently sized areas may be mapped to anotherset of differently sized areas.

In another embodiment, one number of regions may be mapped to adifferent number of regions.

In some embodiments, it may be desirable to ensure that neighbouringregions on the input side are still neighbours on the output side, so asto avoid gaps of illumination between two or more regions and thereforeincorrectly reduce or increase the luminance or brightness in that gapbetween the two regions. As an example, if the light distribution schemedictates that one region be translated away from its neighbouringregion, then it should also dictate that the neighbouring region betranslated along with it, either by pure translation, or by scaling, orby a combination of the two.

In an example embodiment, light incident on each of a plurality of areason a PMD is scaled and/or translated by causing the area of the PMD toemulate a lens, a prism, or a combination thereof. The output lightfield is made up of the scaled/translated light.

Some embodiments may include optical switches that may be operated toselectively direct light from one area of the image or light-field to adifferent area or a number of different areas of the image orlight-field. Such optical switches may augment a PMD or replace a PMD insome embodiments. For example, optical switches may be applied toselectively direct light from a light source into one of a plurality ofintegrating rods or optical fibers. The intensity of light being carriedin the integrating rods or optical fibers may be adjusted by settingoptical switches to change which integrating rods or optical fiberscarry light from which light sources.

Another embodiment uses one or more scanning lasers to provide a desiredlight field. For example, a laser beam can be scanned rapidly across anarea, leaving the viewer with the impression that the whole area isilluminated with equal intensity. If the laser beam is a spot, a2-or-more axis scan may be performed. If the laser beam has the shape ofa line, a 1- or more axis scan may be performed. The beam may be widenedby using for example a motorized focusing lens. The power density orlight intensity of various regions of the image can be controlled byvarying one or more of for example the scanning speed, the size of thearea to be scanned, or the density of the scanning pattern.

Application Classes: Display, Projection, Illumination

Several classes of applications may apply the approaches describedherein. These application classes include, but are not limited to lightprojectors that utilize a projector and a screen to form an image on aprojection screen, displays such as televisions and control monitors andgeneral illumination devices such as smart lamps and lighting. Allapplications share, that it is desirable to efficiently illuminate anobject or form an image. Some applications might be static, but otherscan dynamically change the light pattern formed.

Display

Displays can include a projector that is mounted fixed to a surface tobe illuminated. The projector can be controlled in such as way as toform an image on the surface by illuminating a reflective or partiallyreflective surface from the front. It can also be controlled as to forman image by rear-projecting onto a transmissive or partiallytransmissive surface from the rear. Additional optics can be used toallow varying the distance and angle between the screen and theprojector, for example folding mirrors may be used to fold the opticalpath and place the projector very close to the screen. Lenses may beused to magnify, focus and/or distort the image to match the propertiesof the projection screen such as dimensions, curvature and surfaceproperties.

Display—TV

Example use cases of such a display include a system to replacetelevisions and other displays. For example, a light efficientilluminator may be mounted to a retractable or fixed screen using opticsthat allow very close placement of the projector and screen. The entiresystem may be mounted to a wall, like a TV. The light efficientilluminator may be mounted at the bottom relative to the screen or atthe top relative to the screen. The screen maybe flexible, retractableor solid. It maybe incorporated into a building structure, such as aceiling or a wall.

Display—Billboards

Billboard displays, digital and static signs and advertising displaysare other example display systems that may apply the invention. Anexample of a billboard is a digital display used to display a sequenceof advertising patterns on the shoulder of a highway. The viewing angleof such a display may be optimized in a way that all or most of thelight is reflected in the direction of oncoming traffic and not in otherdirections. A system replacing or upgrading existing paper-basedbillboards to display digital content may be desirable.

Display—Flight Simulators

Another example of display systems that may apply this invention areflight simulators, used for on ground flight training of airplanepilots. In such an application it may be desirable to achieve efficientimage formation on a flat or curved screen or display features that arein focus on a volume rather than on a plane.

Projection Systems:

Projection systems a may include a projector and a screen that arealigned so that the projector forms an image on the screen. The screenmaybe flat, curved or of arbitrary shape. The screen may have certainreflectance properties, such as a lambertian reflectance profile or asomewhat directional reflectance profile. The screen might betransparent, or partially transparent. It may also be perforated toallow air and other media to pass through it.

Projection Systems: Cinema and Home Cinema

An example of a projection system that may apply this invention arecinema projectors in which a projector mounted above and behind theaudience forms an image on a flat or curved large front projectionscreen. In such a system it might be desirable to achieve very high peakbrightness levels (luminance) for some or all parts of the image. Anexample of a high peak brightness might be a level above 48 cd/m².Another example of a high peak brightness might be a level above 200cd/m². Another example of a very high peak brightness might be 1000 to10000 cd/m². Generally a high peak brightness is significantly brighterthat the level that a viewer might be adapted to in the viewingenvironment. In such a system it might also be desirable to achieve highcontrast levels (dark compared to bright features). It might bedesirable to achieve such contrast levels either sequentially orsimultaneously. In such a system it might also be desirable to achievevery pure colours.

An example of a system with similar requirements is a home theatreprojector for use in a non-public space and for a smaller audience.

Projection Systems: Planetariums

Another example of a projection system that may apply this invention isa projector in a planetarium. A desirable property of such a systemmight be that it produces very small and very bright highlights, such asstars within an overall dark or dim scene (the night sky). Anotherdesirable property of such a system might be that the black level isclose to or identical to pure black, which means that no or little lightis present in dark areas of the scene. Another property of such a systemmight be that the image is in focus on a non-flat surface, such as thedome inside a planetarium.

Projection Systems: Portable and Personal Projection (Military, Mobile,Etc.)

Another example of a projection system that may apply this invention areportable, mobile or personal projectors. One desirable property of sucha system might be that it is small in size and/or light in weight.Another desirable property of such a system might be that it uses littlepower and/or is an efficient system. Another desirable property of sucha system might be that it can be operated from a portable power sourcesuch as a battery pack or a fuel cell or another type of smallgenerator. Another desirable property of such a system might be that itdoes not need a well controlled environment, for example that it canform clearly readable images in a bright environment such as in sunlights. Another desirable property of such as system is that it is easyto setup, which might include that the systems turns on near instantly,that is it in focus on non-flat surfaces or that it undistorts theimage, even if no calibrated projection screen is present. Anotherexample of such a system overlays information on physical objects, forexample instructions and locations of parts in a machine that requiresfixing.

Projection Systems: Head-Up Display

Another example of a projection system that may apply this invention isa head-up display, a display that presents additional information withinthe field of view of a user, for example on a windshield in a car or onthe windows of an airplane. In such a system it is desirable to be ableto form an image that is clearly viewable in many viewing environments.For example a desirable property of such a system might be that it isnot too bright (blinding) when driving at night, but bright enough to bevisible during sunlight. Another desirable property of such a systemmight be that it is in focus on a non-flat surface. Another desirableproperty of such a system might be that the image projected by it may beof high quality such that artifacts (e.g. raised black level) related tothe image projection do not interfere with the field of view it isoverlaying.

Projection Systems: Architectural Structured Illumination and LargeVenue Shows

Another example of a projection system that may apply this invention isa projector that forms an image on a building or on a differentstructure as part of an artistic light show. Another example is theprojection onto a stadium floor and/or ceiling. In such a system itmight be desirable to be able to form very bright highlights, that aresignificantly brighter that the surrounding light levels and thereforestick out. It might also be desirable for the system to be veryefficient to lower cost of the installation, setup time and coolingrequirement. It might also be desirable for such a system to be able todynamically focus parts of an image or light field onto different planesor objects within a volume.

Illumination and Lighting

Illumination and lighting systems that may apply this invention can beused in applications in which it is desirable to statically ordynamically illuminate objects and not illuminate other objects. In someapplications it might be desirable to use an illumination system withthe capability to illuminate an entire scene or only parts of a scenesimultaneously. Another desirable property of such a system might bethat the illumination system closely or approximately resembles thespectral reflectance properties of the objects that are to beilluminated. In some applications the scene to be illuminated may firstbe analyzed by a camera or other imaging device.

Illumination and Lighting: Automotive

An example of a system that may apply this invention is dynamic or smartautomotive headlights. It might be desirable to illuminate an upcomingroad sign and the road itself brightly, but at the same time not blindupcoming traffic. It might also be desirable to illuminate the roadahead efficiently. Such a system could work in concert with a camerathat analyses the scene ahead and provides a target light profile to theillumination system. Another desirable property of such a system mightbe that the illumination system closely or approximately resembles thespectral reflectance properties of the objects that are to beilluminated.

Illumination and Lighting: Theater Lighting

Another example of an illumination system based on this invention is atheater spot light. Commonly such spotlights create a bright spot thatcan be manually or automatically moved to follow for example an actor ona theater stage. It might be desirable to replace such a system with anillumination system that can illuminate the entire stage and createlocal spot lights dynamically. A desirable property of such a systemmight be that it efficiently illuminates one ore more objects within thescene. A common data projector is an example of a system that canilluminate a large scene, but is not very efficient when illuminatingonly a partial scene or object, because light is blocked in regions inwhich illumination is not required. A system based on this invention maypresent an efficient alternative. Another desirable property of such asystem might be the ability to focus light on objects that are atdifferent distances from the illumination system.

Some light sources, such as lasers light sources, can produce coherentlight, which is a desirable property in some embodiments. However,depending on the brightness requirement of the final system output lightfield, one might require several independent light sources inside thelight module or a different type of light source, such as an LED or abroadband lamp. This might result in a non-coherent light profileimpinging on a dynamically-addressable light redistributor.

FIG. 7 shows an example of a system that does not require coherentlight. The output from light module 704 is shaped and relayed onto adynamically-addressable light redistributor 703 using for example anoptical module 700 comprising for example two focusing lenses 701 and702. The objective may be to form an image in the plane 705, which mightbe a projection screen. Dynamically-addressable light redistributor 703may be configured, for example with a pre-determined phase pattern, toproduce the desired light field at 705.

In some embodiments an output light field may be required with higheruniformity than can be achieved with the available optics including thedynamically-addressable focusing element. In this case additional optics(such as integration rods, integrating chambers, optical fibres, etc.)may be provided to homogenize the intensity profile in one or moresubsections of the light field. A coarse initial light field may beproduced by the system (by an approach using coherent light ornon-coherent light) and presented upon the input port of such additionaloptics.

FIG. 8 shows an example case in which a plurality of light sources arearranged to illuminate a dynamically-addressable focusing element 801.The light re-distribution scheme driving the dynamically-addressablefocusing element causes the dynamically-addressable focusing element todistribute an initial light field onto an array of integration rods. Theoutput light field of the integration rod array is made up of uniformrectangular fields having intensities at least approximately equal tothe total light present at the input port of each integration roddivided by the cross section area of the integrating rod.

For the sake of illustration dynamically-addressable focusing element801 has been sub-divided into two equally sized regions R1 and R2. 810,820 and 830 illustrate example light profiles at various stages of thisoptical system. 810 depicts light profiles 811, 812, 813 and 814 fromLS1 to LS4. 811 and 812 are incident on RE 813 and 814 are incident onR2. Region R1 may be present on integration rod IR1 and region R2 may bepresent on integration rod IR2. Dynamically-addressable focusing element801 re-distributes the incident light.

In this example the light from LS2 is re-distributed to integration rodIR2 along with the light from LS3 and LS4. The light from LS1 is presenton integration rod IR1. 820 depicts the light profile incident on IR1and IR2. 821 is present on IR1 and stems from LS1, 822 is present on IR2and stems from LS2 and LS3 and 823 is present on IR2 and stems from LS4.830 shows the output light profile from the integration rod array, where831 shows the output from IR1 and 832 shows the output from IR2.

FIG. 9 depicts an example framework of a program to be executed on thesystem processor. Its purpose is to compute a light re-distributionscheme (for example in form of a 2D array of phase retardation values)that, then be addressed onto the dynamically-addressable focusingelement to, in combination with the input illumination profile andsystem optics cause an output light profile that closely matches thetarget light field.

FIG. 9 illustrates an iterative method to finding a solution using aminimization or optimization approach based on a forward model and itsinverse. This example shows an iterative method “the optimizer” thatfinds a solution using a minimization or optimization approach based ona forward model and its inverse. An initial guess of the lightre-distribution scheme as well as regularization terms may be utilizedto converge towards a suitable solution in fewer iterations. Systemconstraints may also be supplied. An exit metric, for example themaximum number of iterations, a residual, or a perceptual metric,determines when the program stops and outputs the current solution inform of a light re-distribution scheme.

In alternative embodiments a metric may determine when the currentsolution is ready to be applied to drive a PMD. After this occurs, theprogram may continue iterating to find a better solution. As bettersolutions are obtained they may be applied to drive the PMD. In anexample embodiment, after a set number of iterations (e.g. 3, 4, 10, afew iterations) the current best phase pattern solution may be appliedto the PMD while the computer/algorithm continues to calculate bettersolutions. After another few iterations a new (and better) solution maybe available and can be addressed on the PMD. All this can happen withinfractions of a frame of a video.

An example of a pair of a forward model and its inverse model is atwo-dimensional Fourier transformation and a two-dimensional inverseFourier transformation, which could be used as an approximate model fora coherent, collimated beam of light passing through a lens.

In another example, the forward model may describe the amount ofdeflection caused by a difference in phases or light path lengths ofneighbouring regions on the PMD when illuminated by light, as well asthe resulting intensity profile. Although an exact inverse system modelmay not exist, an iterative optimization approach may be used to solvefor an approximate solution of the phase pattern to be addressed on thePMD.

FIG. 10 depicts another example framework of a program that may beexecuted on the system processor. Its purpose is to compute a lightre-distribution scheme (for example in the form of a 2D array of phaseretardation values) that, when addressed onto thedynamically-addressable focusing element will, in combination with theinput illumination profile and system optics cause an output lightprofile that closely matches the target light field or an output lightfield that may require further optical or other treatment to produce thedesired target output light field.

The method of FIG. 10 creates an approximation of the full target lightfield using a number of component light fields that may be addedtogether. Examples of such components may be rectangular shaped uniformlight fields or Gaussian light fields.

This example shows a procedural approach (non-iterative) that has accessto a set of features, such as virtual optical elements (described by forexample their shapes, sizes and positions on a PMD and its resultinglight field which can be either pre-computed or provided in functionalform as well as description of parameters that may be varied by theprogram). Each such feature, when applied to the PMD results in arelated output component light field. A light field analyzer comparessuch features, in particular the resulting output light fields to thetarget light fields and determines a set of feature parameters for aplurality of features. These are combined into a final lightre-distributing scheme in the pattern building block.

As an example, one virtual optical element may produce a Gaussianintensity profile as its output. The peak-intensity and full-widthhalf-maximum may be described directly when the virtual optical elementis formed on the PMD. A superimposed set of such Gaussians may resemblethe target light-field when relayed on a display screen.

FIG. 11 shows three examples of optics and the light field that may beused where coherent light is present on the dynamically-addressablefocusing element and a system transfer function (forward model), such ascan be described by a two dimensional Fourier transformation is desired.Example a) uses coherent and collimated light present on thedynamically-addressable focusing element followed by a field lens tofocus light. Example b) achieves a similar result, however the fieldlens is represented as a path length varying pattern that can besuperimposed with the desired light redistribution scheme. Example c)provides similar results, utilizing a slightly converging beam.

Common for all three implementations is that coherent or partiallycoherent light from a light module 1101 is expanded using expansion andbeam shaping optics 1102 and the light is incident on the PMD 1105. 1102may comprise two or more focusing elements 1103 and 1104. In 1110, thelight incident on 1105 is collimated. The output light from 1105 isfocused by focusing lens 1106 onto 1107 in the case when 1105 does notchange phase or changes phase uniformly across the device.

In 1120, the light incident on 1105 is collimated. The output light from1105 is focused onto 1107 in the case when the phase retardation patternon 1105 resembles a lens or an approximation thereof.

In 1130, the light incident on 1105 is not collimated, but converging tofocus at 1107 in the case when 1105 does not change phase or changesphase uniformly across the device.

In all three implementations, 1105 is configured in such a way as toproduce an approximation of a desired target light field at 1107, forexample at a projection screen.

FIG. 12 shows an example of variable scanning optics in which two laserbeams or other light beams collectively scan an area. This scheme may beexpended to any suitable number of light beams. Each light beam scans aregion within the area. By altering the sizes of the scanned regions theluminance within each region may be varied. The dimensions of thescanned regions may be changed, for example, the scanned regions may berectangular and the lengths and widths of the scanned regions may bechanged, thereby altering the areas of the scanned regions. The scanningspeed and pattern may remain fixed as the size and/or shape of a scannedregion is changed or may be altered. For example, scanning speed may bereduced when the scanned region is smaller and increased when thescanned region is larger.

In the example implementation illustrated in FIG. 12 a plurality oflight sources such as lasers LS1 and LS2 are used in conjunction withfor example steerable MEMS mirrors that allow the beams to be scannedacross different areas. In this example LS1 illuminates Area 1 and LS2illuminates Area 2. Together, LS1 and LS2 illuminate the entire imagingarea 1201.

LS1 provides a beam of light incident on a two-axis scanning mirror 1204that scans the beam 1203 over Area 1. LS2 provides a beam of lightincident on a two-axis scanning mirror 1205 that scans the beam 1202over Area 2. The beams 1203 and 1202 may be scanned over the respectiveareas in a line-by-line fashion or any other desired path. In thisexample Area 1 is of a different size than Area 2. Therefore thescanning speed of 1202 maybe increased order to scan over Area 2 in thesame time required for 1203 to scan Area 1. Provided that LS1 and LS2are of same or similar intensities, the resulting average intensity ofArea 1 may be higher than that of Area 2. Other scanning patterns, scanspeeds, pattern densities, light source modulation or intensities may beutilized to achieve a similar effect.

In some embodiments, an image data processor divides the image intoequal area sections, each section being illuminated by one light sourceeach with a modulated amplitude according the requirements of theindividual areas.

In some embodiments, the illumination comes in the form of a uniformfield that covers its whole subsection. In some embodiments, the imagesections comprise differently sized areas of equal light energy of oneor more of the present frequencies in the image (colours), so that eachsub-section and frequency may reach different peak amplitudes.

In some embodiments, the illumination comes in the form of a spot lightsource which is scanned across its subsection of the screen, for exampleusing a laser and a mirror mounted on a rotational 2-axis mirror.

In some embodiments, the illumination comes in the form of a line lightsource which is scanned across its subsection of the screen, for exampleusing a laser and a mirrors mounted on a rotational 1-axis mirror.

In some embodiments, each image subsection is illuminated with asequence of random illumination patterns, each generated by an imagedata processor that ensures that the resulting image is in accordancewith the input image data.

In some embodiments, the projection system comprises differently sizedareas of equal light energy of one or more of the present frequencies inthe image (colours). Each subsection goes through spatial lightamplitude modulation, for example by an SLM.

In some embodiments, the projection system comprises differently sizedareas of equal light energy of one or more of the present frequencies inthe image (colours). Within each subsection, the scanning is done atvariable speed, but with a constant intensity, so that the perceivedsubsection varies in intensity. A controller may be configured to ensurethat each subsection while scanned at variable speeds within eachsubsection still completes to total subsections in the same amount oftime.

In some embodiments the required scanning speed is high and instead of amotorized mirror, diffractive dynamic elements such as acousto-opticalmodulator or acousto-optical deflectors are applied to scan the lightused.

FIG. 13 shows example variable laser scanning optics which is analternative to the implementation depicted in FIG. 12. In FIG. 13, eachlight source is relayed onto a respective area not as a scanning beam,but as a complete field expanded using beam shaping and expansionoptics.

Beam LS1 is shaped and expanded using the asymmetric beam expander ABE1.The expanded light profile is then steered using a two-axis scanningmirror 1302 onto Area 1 of the image. Similarly, LS2 is shaped andexpanded using the asymmetric beam expander ABE2. The expanded lightprofile is then steered using a two-axis scanning mirror 1301 onto Area2 of the image.

Asymmetric beam expander ABE2 is explained in 1310 and 1320. 1310 showshow the incident beam is expanded by different amounts in twoperpendicular directions using the cylindrical lenses 1311 and 1312. Inits default configuration, the input beam profile which may be in theform of a focused spot, will result in the output light profile being asquare and uniform intensity beam. If a rectangular shape is required,one or both of the cylindrical lenses 1311 or 1312 may be moved by forexample a motor along the optical axis of this system. Moving 1311 willchange the square shape 1321 into shape 1322. Likewise, moving 1312 willchange the square shape 1323 into shape 1324. If both lenses 1311 and1312 are moved, the output light profile may be as depicted in 1325. Inaddition to shaping the individual beams, the output light profiles fromABE1 and ABE2 may be steered by 1301 and 1302 so that their combinedillumination from 1300 and 1303 fills the complete image.

FIG. 14 shows an example application of optical switches to remove lightfrom a first area and direct that light to a second area. The first areabecomes dimmer and the second area becomes brighter as a result.

In FIG. 14, a plurality of light sources LS1 to LSn provide input beamsonto an array of input ports 1400, for example via optical fibres. Thelight from each input port is present on device 1403 comprising one ormore elements such as 1405 that steers the light. 1405 could for examplebe a two-axis controllable mirror. This light can furthermore be relayedoff a secondary device 1404 comprising one or more elements such as 1406that steers the light towards an output array of output ports. In oneembodiment, the input light beams provided by the light sources arecollimated and remain collimated at the output port 1401.

FIG. 15 depicts an implementation of a light-efficient illuminator usingoptical switches and a clean-up module such as an array of integratingrods IR1, . . . , IRn 1510, a plurality of light sources LS1 to LSnprovide input beams onto an array of input ports 1500, for example viaoptical fibres. These beams are further directed onto a clean-up module,for example an array of integration rods IR1 to IRn. In this example,LS1 and LS2 are incident on IR1, LS3 and LS4 are incident on IR2 and soforth. 1511 shows the incident light profile on IR1 and IR2. 1513 is thelight stemming from LS1. 1514 is the light stemming from LS2. 1515 isthe light stemming from LS3. 1515 is the light stemming from LS4. Theremaining integrating rods in this system can be described in a similarmatter. 1512 shows the output light profile from integration rods IR1and IR2, and illustrates how the non-uniform input profiles have beenmade uniform. The average intensity of 1517 is approximately that of theaverage intensity of 1513 and 1514 combined or integrated. The averageintensity of 1518 is approximately that of the average intensity of 1515and 1516 combined or integrated.

In 1520 shows the same system with steering. In this example, LS2 issteered away from IR1 and into IR2, thereby reducing the intensityoutput from IR1 and increasing that of IR2. 1521 shows the input lightprofiles onto IR1 and IR2. 1523 shows the input light profile from LS1onto IR1. 1524 shows the combined light input from LS2, LS3 and LS4 ontoIR2.

1522 shows the output light profile from integration rods IR1 and IR2.The average intensity of 1525 is approximately that of the averageintensity of 1523. The average intensity of 1526 is approximately thatof the average intensity of 1524 combined or integrated.

FIG. 16 illustrates an embodiment which applies a refinement module (forexample a DMD) to fine tune a light field.

The output light field of a device as described herein may be used as acontrollable light source to illuminate a refinement stage which mayinclude an imaging device, like a DMD, an LCD, or an LCoS. The imagingdevice may comprise an amplitude modulating SLM, a PMD, a diffusivedevice, like a DMD, an LCD, or an LCoS. Refinement may provide a refinedoutput that may improve the quality of the output light-field of thecomplete system. The refinement module may comprise a de-specklingmodule, a polarization-varying device, a colour-enhancing device etc.

The refined output light-field may be imaged or relayed onto aprojection screen or the like in the usual manner. As shown in FIG. 16,a light-efficient illuminator may be combined with one or severalrefinement modules which may be arranged in serial and/or parallelconfigurations.

For example, the refinement module may provide high spatial frequencydetail to the final light-field that can now be imaged or relayed onto aprojection screen or the like on top of the output light-field from thelight efficient illuminator.

Another use of the refinement module is to minimize visual artifactsintroduced by the light efficient illuminator. In some implementations,these improvements may be executed in accordance with analysis of thelight efficient illuminator output as well as models of the human visualsystem such as colour appearance models, visual difference predictors orother models.

In a preferred embodiment, the dynamic light efficient illuminatorprovides a first low quality version of the target light field, forexample low spatial resolution colour image, as input to the refinementmodule. The refinement module may comprise a single spatial amplitudemodulator of high spatial resolution relative to the colour image andprovide luminance modulation only. The combined output may be differentfrom the target light field in an absolute sense, but perceptuallycomparable.

FIG. 17 depicts an example optical implementation as well as lightprofiles present at various stages of a light efficient illuminator witha refinement module. The output light field 1700 from the lightefficient illuminator is relayed onto the refinement module 1703 via forexample a relay lens system 1701. 1704 shows an example output lightprofile produced by the light efficient illuminator. 1705 shows anexample light profile incident on the refinement module 1703. In someembodiments it may be preferable not to focus the output of 1700 exactlyonto 1703, but instead blur or spatially low-pass filter 1704 by moving1703 by an amount 1702.

FIG. 18: illustrates an embodiment comprising a refinement module (forexample a DMD) and a clean-up module (for example an integration rodarray).

Certain applications may require that the output light field be of highfidelity, uniformity, or predictability. In some cases an output lightfield may include undesirable sharp spikes or other optical noise notspecified for the target light field. One could reduce these issues byproviding an array of homogenizing elements such as an array ofindependent integrating rods or a bundle of fibers or a number ofoptical waveguides between the illuminator and the refinement module.The spatially-varying output light-field is then reduced to a number ofknown intensity profile regions matching the number of homogenizingelements (e.g. fibers or integrating rods) used. The benefits ofsteering light into a number of integrating rods, fibres or regions mayinclude enhanced contrast, improved black-levels, higherpeak-intensities, and/or improved predictability of the output lightprofile. Additionally, as light that would otherwise pass through onehomogenizing element can be redirected into another homogenizingelement, the resulting peak luminance can in the best case be increasedby approximately the number of regions (e.g. all or most of the lightcould be directed into one of the homogenizing elements). By contrast,in a system that does not redirect light but merely attenuates light toachieve desired luminance levels, the peak luminance is limited to theglobal luminance level that the system's light module can provide.

FIG. 19 depicts an example implementation of a light-efficientilluminator combined with a clean-up module and a refinement module. Theoutput of the light-efficient illuminator 1900 is relayed using forexample a lens 1901 onto an array of integration rods 1902, IR1, IR2,IR3 and IR4. The output 1903 of the array of integration rods is furtherrelayed onto 1905 using for example a lens 1904. 1905 represents therefinement module, for example in the form of an amplitude-modulatingSLM. The output of 1905 is relayed onto for example a projection screen1912 using for example a lens 1911. An example output light-field from1900 is depicted in 1907. The light profile incident on 1905 may beblurred using a small offset 1906, turning the light profile 1908 at1903 into 1909 at 1905. 1910 shows the final output light-field from thecomplete system, where 1913 represents a relayed version of 1909modulated to 1914 by 1912.

Projection systems as described herein may comprise multiple stages ofmodulation. One or more stages may modulate phase of the light, and/orone or more other stages modulate the light's amplitude, and/or one ormore other stages modulate the light's frequency, and/or one or moreother stages modulate the light's polarization. For example, oneprojection system could have two spatial amplitude modulators, and onespatial phase modulator.

Such stages may be arranged to process light in a serial or parallelfashion, or a mix thereof. In the parallel case, different light fieldscan be combined by a beam-splitter or beam-combiner, for example adichroic mirror in order to combine light of different frequencies, or apolarizing beam-splitter in order to combine light with differentpolarizations. For the serial case, the output of one stage is used asthe input for the next stage. For example, one system can use afrequency modulator in parallel with a phase modulator, both of whichare placed in series with a spatial amplitude modulator. The twoparallel stages can be combined by a beam-combiner before they arerelayed onto the spatial amplitude modulator which is placed in serieswith the parallel stage.

FIG. 20 shows intensity-time and intensity-location plots in a systememploying a color field sequential scheme. 2010 indicates an example ofa desired spatial cross-sectional light profile in a three-primarylight-efficient illuminator made up from a red (2011), a green (2012)and a blue (2013) channel. Since the light-efficient illuminatorachieves its target light profile by re-distributing light rather thanexclusively amplitude-modulating light (by attenuation), the totalamount of light required by each light source is proportional to theintegrated light of each colour channel light profile, 2011, 2012, and2013 respectively for R, G, and B. In this particular example, one caninspect the light profiles 2011, 2012 and 2013 and deduce that a lesseramount of green light is required, an intermediate amount of red lightis required, and a largest amount of blue light is required.

The total amount of light for each color within one video frame in acolor field sequential drive scheme can either be modulated at thesource by reducing the intensity of individual light sources over afixed period of time as depicted in 2000, or by a fixed intensity overvarying amounts of time as depicted in 2020 or by a combination of thesetwo schemes, not depicted here. In 2000 the intensity modulatingapproach is depicted for red by 2001, green by 2002 and blue by 2003. In2020 the time modulating approach is depicted for red by 2021, green by2022 and blue by 2023.

Interpretation of Terms

Unless the context clearly requires otherwise, throughout thedescription and the

-   -   “comprise”, “comprising”, and the like are to be construed in an        inclusive sense, as opposed to an exclusive or exhaustive sense;        that is to say, in the sense of “including, but not limited to”;    -   “connected”, “coupled”, or any variant thereof, means any        connection or coupling, either direct or indirect, between two        or more elements; the coupling or connection between the        elements can be physical, logical, or a combination thereof;    -   “herein”, “above”, “below”, and words of similar import, when        used to describe this specification, shall refer to this        specification as a whole, and not to any particular portions of        this specification;    -   “or”, in reference to a list of two or more items, covers all of        the following interpretations of the word: any of the items in        the list, all of the items in the list, and any combination of        the items in the list;    -   the singular forms “a”, “an”, and “the” also include the meaning        of any appropriate plural forms.

Words that indicate directions such as “vertical”, “transverse”,“horizontal”, “upward”, “downward”, “forward”, “backward”, “inward”,“outward”, “vertical”, “transverse”, “left”, “right”, “front”, “back”,“top”, “bottom”, “below”, “above”, “under”, and the like, used in thisdescription and any accompanying claims (where present), depend on thespecific orientation of the apparatus described and illustrated. Thesubject matter described herein may assume various alternativeorientations. Accordingly, these directional terms are not strictlydefined and should not be interpreted narrowly.

Embodiments of the invention may be implemented using specificallydesigned hardware, configurable hardware, programmable data processorsconfigured by the provision of software (which may optionally comprise“firmware”) capable of executing on the data processors, special purposecomputers or data processors that are specifically programmed,configured, or constructed to perform one or more steps in a method asexplained in detail herein and/or combinations of two or more of these.Examples of specifically designed hardware are: logic circuits,application-specific integrated circuits (“ASICs”), large scaleintegrated circuits (“LSIs”), very large scale integrated circuits(“VLSIs”), and the like. Examples of configurable hardware are: one ormore programmable logic devices such as programmable array logic(“PALs”), programmable logic arrays (“PLAs”), and field programmablegate arrays (“FPGAs”)). Examples of programmable data processors are:microprocessors, digital signal processors (“DSPs”), embeddedprocessors, graphics processors, math co-processors, general purposecomputers, server computers, cloud computers, mainframe computers,computer workstations, and the like. For example, one or more dataprocessors in a control circuit for a device may implement methods asdescribed herein by executing software instructions in a program memoryaccessible to the processors.

Processing may be centralized or distributed. Where processing isdistributed, information including software and/or data may be keptcentrally or distributed. Such information may be exchanged betweendifferent functional units by way of a communications network, such as aLocal Area Network (LAN), Wide Area Network (WAN), or the Internet,wired or wireless data links, electromagnetic signals, or other datacommunication channel.

For example, while processes or blocks are presented in a given order,alternative examples may perform routines having steps, or employsystems having blocks, in a different order, and some processes orblocks may be deleted, moved, added, subdivided, combined, and/ormodified to provide alternative or subcombinations. Each of theseprocesses or blocks may be implemented in a variety of different ways.Also, while processes or blocks are at times shown as being performed inseries, these processes or blocks may instead be performed in parallel,or may be performed at different times.

In addition, while elements are at times shown as being performedsequentially, they may instead be performed simultaneously or indifferent sequences. It is therefore intended that the following claimsare interpreted to include all such variations as are within theirintended scope.

Software and other modules may reside on servers, workstations, personalcomputers, tablet computers, image data encoders, image data decoders,PDAs, video projectors, displays (such as televisions), digital cinemaprojectors, media players, and other devices suitable for the purposesdescribed herein.

The invention may also be provided in the form of a program product. Theprogram product may comprise any non-transitory medium which carries aset of computer-readable instructions which, when executed by a dataprocessor, cause the data processor to execute a method of theinvention. Program products according to the invention may be in any ofa wide variety of forms. The program product may comprise, for example,non-transitory media such as magnetic data storage media includingfloppy diskettes, hard disk drives, optical data storage media includingCD ROMs, DVDs, electronic data storage media including ROMs, flash RAM,EPROMs, hardwired or preprogrammed chips (e.g., EEPROM semiconductorchips), nanotechnology memory, or the like. The computer-readablesignals on the program product may optionally be compressed orencrypted.

In some embodiments, the invention may be implemented in software. Forgreater clarity, “software” includes any instructions executed on aprocessor, and may include (but is not limited to) firmware, residentsoftware, microcode, and the like. Both processing hardware and softwaremay be centralized or distributed (or a combination thereof), in wholeor in part, as known to those skilled in the art. For example, softwareand other modules may be accessible via local memory, via a network, viaa browser or other application in a distributed computing context, orvia other means suitable for the purposes described above.

Where a component (e.g. a software module, processor, assembly, device,circuit, etc.) is referred to above, unless otherwise indicated,reference to that component (including a reference to a “means”) shouldbe interpreted as including as equivalents of that component anycomponent which performs the function of the described component (i.e.,that is functionally equivalent), including components which are notstructurally equivalent to the disclosed structure which performs thefunction in the illustrated exemplary embodiments of the invention.

Specific examples of systems, methods and apparatus have been describedherein for purposes of illustration. These are only examples. Thetechnology provided herein can be applied to systems other than theexample systems described above. Many alterations, modifications,additions, omissions, and permutations are possible within the practiceof this invention. This invention includes variations on describedembodiments that would be apparent to the skilled addressee, includingvariations obtained by: replacing features, elements and/or acts withequivalent features, elements and/or acts; mixing and matching offeatures, elements and/or acts from different embodiments; combiningfeatures, elements and/or acts from embodiments as described herein withfeatures, elements and/or acts of other technology; and/or omittingcombining features, elements and/or acts from described embodiments.

It is therefore intended that the following appended claims and claimshereafter introduced are interpreted to include all such modifications,permutations, additions, omissions, and sub-combinations as mayreasonably be inferred. The scope of the claims should not be limited bythe preferred embodiments set forth in the examples, but should be giventhe broadest interpretation consistent with the description as a whole.

What is claimed is:
 1. An image projection system to display imagesspecified by image data, the image data specifying luminance that variesfrom place to place on the images such that the images have darker areasand lighter areas, the projection system comprising: a phase modulatorcomprising a number of elements that are controllable to impart aselective phase delay in light passing through each element; a controlcircuit configured to control the elements of the phase modulator tocreate a lens with a phase pattern that directs incident light to form alight field corresponding to the varying luminance of the image data andto focus the directed light onto an image plane; a light source arrangedto illuminate the phase modulator with the incident light; and opticsarranged to project light from the image plane onto a surface wherein:the control circuit is configured to determine a two-dimensional patternof phase delay values for the phase modulator based on a modeldescribing an amount of deflection caused by a difference in phase ofneighboring regions of the phase modulator when illuminated by theincident light.
 2. The image projection system according to claim 1,further comprising a refinement module positioned before the optics thatproject light onto the surface and that comprises a spatial lightmodulator arranged to further spatially modulate light focused at theimage plane by the phase modulator.
 3. The image projection systemaccording to claim 2 wherein the spatial light modulator comprises areflective-type spatial light modulator.
 4. The image projection systemaccording to claim 2 wherein the spatial light modulator comprises atransmissive-type spatial light modulator.
 5. The image projectionsystem according to claim 2 wherein the refinement module comprises anarray of light-integrating elements.
 6. The image projection systemaccording to claim 2 wherein the refinement module comprises a pluralityof integration elements arranged between the phase modulator and theoptics to project light from the image plane onto the surface.
 7. Theimage projection system according to claim 1, wherein the image plane ispositioned at a distance before the refinement module to blur the lightfield before the refinement module.
 8. The image projection systemaccording to claim 1 wherein the image data comprises video data and thecontrol circuit is configured to control the phase modulator to, insequence, vary the phase delay of the elements of the phase modulator togenerate light fields corresponding to images in the video data.
 9. Theimage projection system according to claim 1 wherein the control circuitis configured to perform a first number of iterations to yield aninitial solution of a pattern of phase delay values for the phasemodulator while controlling the phase modulator according to the initialsolution and to determine an additional solution of the pattern of phasedelay values to subsequently control the phase modulator to create thelight field corresponding to the image data.
 10. The image projectionsystem according to claim 1 wherein the control circuit is configured tocontrol the phase modulator to yield the lens having a configurationsuch that the incident light interacting with a first area of the lensis shifted in the light field so as to overlap with light that hasinteracted with a second area of the lens distinct from the first area.11. The image projection system according to claim 1 comprising acollimator arranged to collimate the incident light ahead of the phasemodulator.
 12. The image projection system according to claim 1 whereinthe light source is a controllable-intensity light source and thecontrol circuit is connected to control the intensity of thecontrollable-intensity light source.
 13. The image projection systemaccording to claim 12 wherein the controllable-intensity light sourcecomprises a plurality of variable-intensity light sources and thecontrol circuit is connected to separately control the intensity oflight emitted by different ones of the plurality of variable-intensitylight sources.
 14. The image projection system according to claim 13wherein the variable-intensity light sources each illuminate a distinctarea of the phase modulator.
 15. The image projection system accordingto claim 13 wherein the variable-intensity light sources compriselasers.
 16. The image projection system according to claim 1 wherein thevariable-intensity light source is a broadband light source that emitsbroadband light.
 17. The image projection system according to claim 16wherein the variable-intensity light source is a white light source thatemits white light.
 18. The image projection system according to claim 1wherein the light source emits coherent light or partially coherentlight.
 19. The image projection system according to claim 1 wherein thecontrol circuit is configured to repeat steps of: controlling the phasemodulator to provide the lens having a configuration based on the imagedata; illuminating the phase modulator with the incident light andallowing the incident light to interact with the phase modulator toyield a modified light field; and projecting the modified light fieldonto a surface for each of a plurality of different colours of theincident light in a time-multiplexed manner.
 20. The image projectionsystem according to claim 19 wherein the control circuit is configuredto, within one frame, control on-times for the different colours of theincident light to be different.
 21. The image projection systemaccording to claim 1 wherein the light source is one of a plurality oflight sources and the phase modulator is one of a plurality of phasemodulators and each of the plurality of light sources is arranged todirect incident light onto a corresponding one of the plurality of phasemodulators, the image projection system further comprising a colorre-combiner arranged to combine the resulting plurality of modifiedlight fields to yield a colour image.
 22. The image projection systemaccording to claim 1, wherein the forward and inverse model comprisetwo-dimensional Fourier and inverse Fourier transforms of the imagedata.
 23. The image projection system according to claim 1, furthercomprising a clean-up module positioned before the optics that projectlight onto the surface and that comprises one or more elements selectedfrom an array of independent integrating rods, bundle of fibers,diffusers and optical waveguides.
 24. The image projection systemaccording to claim 23, wherein the clean-up module is positioned beforea refinement module that is positioned before the optics that projectlight onto the surface and that comprises a spatial light modulatorarranged to further spatially modulate light focused at the image plane.